Facial Feature Extraction using Eigenspaces and Deformable Graphs

In model-based coding of image sequences containing human faces, eg videophone sequences, the detection and location of the face as well as the extraction of facial features from the images are crucial. The facial feature extraction can be regarded as an optimization problem, searching the optimum adaptation parameters of the model. The optimum is defined as the parameter set describing the face with the minimum distance to a face space. There are different approaches to reduce the high computational complexity, and here a scheme using deformable graphs and dynamic programming is described. Experiments have been performed with promising results. 1. MODEL-BASED CODING Since the major application of the techniques described in this document is model-based coding, an introduction to that topic will follow here. For more details, see eg [8] or [12]. The basic idea of model-based coding of video sequences is illustrated in Figure 1. At the encoding side of a visual communication system (typically, a videophone system), the image from the camera is analysed, using computer vision techniques, and the relevant object(s), for example a human face, is identified. A general or specific model is then adapted to the object, usually the model is a wireframe describing the 3-D shape of the object. Instead of transmitting the full image pixel-bypixel, or by coefficients describing the waveform of the image, the image is handled as a 2-D projection of 3-D objects in a scene. To achieve this, parameters describing the object(s) are extracted, coded and transmitted. Typical parameters are size, position, shape and texture. The texture can be compressed by some traditional image coding technique, but specialized techniques lowering the bit-rate considerably for certain applications have recently been published [13]. At the receiver side of the system, the parameters are decoded and the decoder’s model is modified accordingly. The model is then synthesized as a visual object using computer graphics techniques, eg a wireframe is shaped according to the shape and size parameters and the texture is mapped onto its surfaces. In the following images, parameters describing the change of the model are transmitted. Typically, those parameters tell how to rotate and translate the model, and, in case of a non-rigid object like a human face, parameters describing motion of individual vertices of the wireframe are transmitted. This constitutes the largest gain of the model-based coding, since the motion parameters can be transmitted at very low bit-rates [2]. Definitions for coding and representation of parameters for model-based coding and animation of human faces are included in the newly set international standard MPEG-4 [10, 11]. 1.1. Components of a Model-Based Coding System To encode an image sequence in a model-based scheme, we need first to detect and locate the face. This can be done by eg colour discrimination, detection of elliptical objects using Hough-transforms, connectionist/neural network methods, or statistical pattern matching. Then, the facial features need to be extracted. This means to find the positions of a set of points or conE nco der D ecod er Channel M odel M odel Origina l